On Estimating Recommendation Evaluation Metrics under Sampling

Since the recent studies (KDD'20) done by Krichene and Rendle on sampling based top-k evaluation metric for recommendation, there have been a lot of debate validity using evaluating recommendation algorithms. Though their work Li et. al. proposed some basic approach mapping metrics to counter-part in global which uses entire dataset, is still lack understanding how should be used evaluation, approaches either are rather ad-hoc or can only simple metrics, like Recall/Hit-Ratio. In this paper, we introduce principled derive estimators sampling. Our utilize weighted MLE maximal entropy recover rank distribution then that estimation. The experimental results shows significant advantages our algorithms metrics.